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Description 

[0001] The present invention relates generally to 
methods and systems for controlling inputs to display 
screens involving tracking the attitude of a device and 
more particularly to tracking the attitude of a device in 
order to remotely control a device or process, such as 
a cursor of a video display. 

[0002] There are applications in which video systems 
require that a person interact with information presented 
on a display screen. At times, the interaction is to occur 
while the person is situated at a* distance from the dis- 
play screen. As will be described more fully below, the 
interaction may be accomplished by remotely control- 
ling a screen cursor in one of a variety of manners. The 
interactions may include selecting from a variety of 
choices presented as a screen menu, or "typing" text 
using an on-screen keyboard. Examples of remote in- 
teractive video systems (RIVS) include interactive tele- 
vision (ITV), TV-style Internet browsers, and confer- 
ence-room video projectors. 

[0003] One key component of a RIVS is the "pointing" 
device for controlling the on-screen cursor. The pointing 
device fulfills a function analogous to that which mice, 
trackballs, and graphic tablets perform for computers. 
However, the environment for RIVS presents difficulties 
that are typically not encountered in operation of a com- 
puter. For example, an operator of a RIVS is typically 
further away from the controlled device than is the op- 
erator of a computer. As another example, the operator 
of a RIVS is more likely to be in an unstructured imme- 
diate environment, e.g., an ITV operator seated across 
a living room from a television set. In many situations, 
the environment precludes use of conventional compu- 
ter pointing devices, such as mice. Moreover, a RIVS is 
rarely equipped with a keyboard, so that the pointing de- 
vice may have to accommodate the extra burden of pro- 
viding a text entry. 

[0004] There are a number of known pointing devices 
for a RIVS. Most of the known pointing devices imple- 
ment some variation of a four-key cursor pad on a hand- 
held controller. The four-key cursor pad is manipulated 
to step the screen cursor up, down, left or right among 
various menu choices. Such interfaces emulate the 
computer keyboard cursor keys used with old-style tex- 
tural interfaces. However, these interfaces are typically 
much slower and less intuitive to use than computer 
mice and other pointing devices developed for modern 
graphical software interfaces. 

[0005] In an effort to improve upon cursor control with- 
in the RIVS environment, more advanced computer 
pointing devices of mice and trackballs have been 
adapted. In one adaptation, a miniature trackball is 
mounted atop a controller, with the trackball being op- 
erated by the person's thumb. The trackball controller is 
faster than the use of cursor keys and facilitates diago- 
nal moves. Unfortunately, the trackball may require re- 
peated strokes to accomplish large cursor movements 



and, in general, thumb control taxes the user's thumb 
dexterity. For example, it is difficult to trace the cursor 
in a circle on the display screen. 
[0006] The use of a mouse for ITV cursor control has 

5 been demonstrated. The advantage of the mouse is that 
it provides excellent and intuitive cursor control. The 
concern is that there may not be a suitable planar oper- 
ating surface that is convenient to the operator. 
[0007] A further refinement in the RIVS pointing art is 

10 the use of devices that enable control of a cursor by 
merely gesturing with a controller. These devices may 
measure the attitude, i.e. pitch, yaw, and possibly roll, 
of the controller. A first category of such an approach 
employs light beams to measure attitude. PCT Interna- 
ls tional Publication Number WO 95/19031 describes a 
system for determining the pointing orientation of a re- 
mote unit relative to a fixed base unit. The fixed base 
unit includes one or more light sources for emitting a 
light beam. The emitted light is polarized in at least one 

20 predetermined orientation. The movable remote unit in- 
cludes a photodetector for detecting the polarized emit- 
ted light. The attitude of the movable remote unit may 
be determined by measuring the intensity of received 
light from various directions. 

25 [0008] Another implementation of the emitted-light 
category of measuring attitude is one in which an infra- 
red (IR) signal is beamed from the area of the video dis- 
play. The IR signal is defocused and is imaged onto a 
quad photodiode array in the controller. The relative sig- 

30 nal amplitudes from the four photodiodes may be used 
to determine the relative orientation of the controller to 
a line drawn from the display. One concern is that the 
system may undesirably flood the room with intense IR, 
rendering other nearby IR-coupled appliances (e.g., a 

35 VCR controller) inoperative. A second concern is that 
the limited range of transmission of defocused IR sig- 
nals may render this system of measuring attitude un- 
reliable when the controller is more than a relatively 
short distance from the video display. 

40 [0009] A second category of devices that measure at- 
titude of the controller is one in which inertia! navigation 
principles are employed. Gyroscopes or encoded gim- 
balled masses establish inertial frames in the control- 
lers, against which attitude changes can be measured. 

45 The attitude information may then be transmitted to the 
video display via a radio-frequency link to a small dipole 
antenna affixed atop the video display. 
[0010] The third category is related to the first cate- 
gory. A hand-held object that provides cursor control has 

50 a number of light sources mounted on one surface. A 
single electronic camera is directed to capture images 
of the light sources mounted on a hand-held object. Lo- 
cations of the images of the light sources are detected 
in each camera image, and a computer is used to de- 

55 termine the attitude of the light-emitting hand-held ob- 
ject. Such a device is described in U.S. Pat. No. 
5,388,059 to DeMenphon. 

[001 1] A closely related need exists in the field of vir- 
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tual reality. In games, simulations, and other visualiza- 
tion situations, it is often necessary to encode the atti- 
tude of a user's head, or other body part. In many cases, 
systems for encoding head pitch and yaw may be ap- 
plied to RIVS controllers, and vice versa. One known 
virtual reality system encodes pitch and yaw by means 
of instrumented compasses and gravimeters. 
[001 2] While the known cursor control devices and at- 
titude-determining systems operate adequately for their 
intended purposes, each is associated with a concern 
or a problem. Operation may be slow or tedious, or may 
require use of a specific operating surface. Devices and 
systems that include IR radiation may adversely affect 
operation of other devices. Attitude-sensing devices 
that are based on gravity may have difficulty in distin- 
guishing tilting from transverse acceleration, thereby 
rendering control erratic. This last problem conceivably 
could be solved by gyro stabilization, but the cost and 
power consumption make this solution unattractive. 
Known systems that utilize light detection require adding 
a second contrivance at the display, again adding addi- 
tional cost. 

[0013] US-A-5,450,126 discloses image blur correct- 
ing apparatus for a video camera incorporating a moving 
vector detector circuit which determines the attitude of 
the camera in the x- and y- directions so as to compen- 
sate for camera shake. 

[0014] What is needed is a method and a system for 
reliably tracking attitude of a device which are cost effi- 
cient when used in controlling a screen cursor or when 
used in other remote interactive video applications. 
[001 5] A method according to the present invention is 
defined in claim 1. 

[001 6] A system according to the present invention is 
defined in claim 7. 

[0017] In embodiments of the invention, correlation of 
successive images acquired by means of a two-dimen- 
sional array of photosensors is used as a basis for track- 
ing attitude of a device to which the array is affixed. In 
the preferred embodiment, the device is a hand-holda- 
ble member, such as a controller for maneuvering a cur- 
sor on a display screen of a video set. Based upon the 
step of correlating images to detect differences in loca- 
tion of imaged features that are common to a succession 
of images, the system generates an attitudinal signal in- 
dicative of any changes in angular orientation during the 
time period of acquiring the images. That is, the attitu- 
dinal signal is determined by the pitch and yaw, and op- 
tionally the roll, of the device that bears the array of pho- 
tosensors. Since the acquired images need not be re- 
lated to that which is being controlled, e.g. a screen cur- 
sor, the device can face in any direction during the con- 
trol process. Moreover, it is not necessary to provide a 
dimensional one-to-one correspondence of angular dis- 
placement of the device and travel of that which is being 
controlled. Within cursor control, for example, the con- 
troller may be directed arbitrarily and relationships of de- 
grees of pitch and yaw to lengths of cursor movement 



may be user-adjustable. 

[001 8] The two-dimensional array of photosensors is 
used to acquire a reference frame for tracking the atti- 
tude of the device. The reference frame is stored and a 
5 second image of features within a field of view of the 
array is acquired. The second image may be considered 
to be a sample image, and the fields of view of the two 
images should be largely overlapping, so that the refer- 
ence and sample frames include a number of common 
features. While not critical, the device includes optics 
which provide a focus nominally at infinity, intentionally 
presenting an off-sharp image to the array of photosen- 
sors. In the application of the device for controlling a 
screen cursor, the representative imaged features will 
typically include windows, lamps, furniture and the dis- 
play screen itself. In any application of the invention, one 
or more stationary sources of light may be specifically 
added within the environment to be imaged, so that suc- 
cessive images of the fixed light are used for the pur- 
pose of correlation. In one implementation of such an 
embodiment, the source of light is an IR emitter and the 
imaging array on the device is provided with IR filtering 
to permit tracking of the attitude of the device. 
[0019] Conceptually, the step of correlating the refer- 
ence frame with a sample frame is one in which one of 
the frames is fixed in position and the other frame is re- 
peatedly shifted to determine which shifted position best 
approximates an alignment of the imaged features that 
are common to the two frames, thereby allowing the de- 
termination of the pitch and yaw of the imaging array 
during the interval between acquiring the two frames. In 
practice, the shifts are performed computationally and 
are shifts of pixel values in which each pixel value is 
indicative of light energy received at a particular photo- 
sensor at a specific time. The correlations may be lim- 
ited to computational shifts of only one pixel for nearest- 
neighbor correlations, or may be multi-pixel computa- 
tional shifts. The nearest-neighbor correlation process 
is often preferred, since it is less computationally com- 
plex, with only the original position and eight computa- 
tional shifts being necessary. Interpolations are then 
performed to determine angular displacements that are 
less than a full pixel. Angular displacement of the device 
about a horizontal axis, i.e. pitch, will result in the ar- 
rangement of pixel values of the reference frame being 
moved upwardly or downwardly. Angular displacement 
of the device about a vertical axis, i.e. yaw, will result in 
the pixel value arrangement being moved to the left or 
to the right. The system detects pitch, yaw and combi- 
nations of pitch and yaw. The attitudinal signal that is 
generated by the system is responsive to the detection 
of such angular displacements. Optionally, roll may also 
be considered. 

[0020] In the application in which the attitudinal signal 
is generated in order to control a screen cursor, the de- 
vice preferably includes a transmitter for wireless trans- 
mission of a cursor-control signal. For example, the sig- 
nal may be transmitted via an infrared beam. Changes 
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in the pitch of the hand-holdable device are then trans- 
lated into vertical movements of the screen cursor, while 
changes in device yaw will move the screen cursor lat- 
erally. In this embodiment, translational movement of 
the device may also be detected and utilized, so that 5 
vertical or horizontal movement of the device translates 
to a corresponding vertical or horizontal movement of. 
the screen cursor. 

[0021] One concern in the implementation of the 
method and system is the effect of the phenomena w 
known in the lens design as curvilinear distortions. Cur- 
vilinear distortions are also referred to as pin-cushion, 
barrel, and perspective distortions. Rectilinear detail is 
compressed at the outer edges of the field by such dis- 
tortion. Curvilinear distortion is particularly pronounced *5 
in simple lenses with wide fields of view, such as the 
lens contemplated for use with the present invention. In 
the invention, the field of view is preferably approximate- 
ly 64°, so that curvilinear distortions will inevitably occur. 
[0022] In the preferred embodiment, the photosen- 20 
sors of the array vary dimensionally in order to define 
an array that is curvilinear, i.e., includes an arcuate outer 
edge. The curvilinear array is dimensioned to compen- 
sate for the curvilinear distortion introduced by the lens 
system. The imaging by the optics is evaluated to char- 25 
acterize the curvilinear distortion, with the array then be- 
ing patterned to offset the distortion. In this manner, the 
arrangement of the photosensor array and the optics 
greatly reduces adverse effects of curvilinear distortion. 
[0023] An advantage of the invention is that device at- 30 
titude may be tracked in a reliable and cost-efficient 
manner. For those applications in which the array-bear- 
ing device is a hand-holdable device, control of a screen 
cursor or the like is economically achieved without a pre- 
mium on dexterity. Moreover, the device does not re- 35 
quire operation on a suitable surface. 
[0024] Fig. 1 is a perspective view of a hand-holdable 
device that enables attitude tracking of the device, in 
accordance with the invention. 

[0025] Fig. 2 is a block diagram of components for 40 
controlling a screen cursor in accordance with the in- 
vention. 

[0026] Fig. 3 is a block diagram of an array of photo- 
sensors and transfer amplifiers of the correlator of Fig. 2. 
[0027] Fig. 4 is an operation view of the steps for cor- 45 
relating successive images in accordance with the in- 
vention. 

[0028] Fig. 5 is a conceptual view of the steps of Fig . 4. 
[0029] Fig. 6 is an embodiment of a computation cell 
of an array of cells for carrying out the steps of Fig. 4. 50 
[0030] Figs. 7-11 illustrate arrays of photosensors and 
computation cells for demonstrating and then reducing 
curvilinear distortions. 

[0031] With reference to Fig. 1 , a hand-holdable con- 
troller device 1 0 is shown as including an integrated cir- 55 
cuit package 1 2 for acquiring and processing image da- 
ta. On the surface of the package is an optoelectric in- 
tegrated circuit chip 14 having a two-dimensional array 



6 

16 of photosensors and having processing circuitry 18. 
The array and processing circuitry are used to acquire 
successive frames of image data, permitting the attitude 
of the device to be tracked. Thus, angular displacement 
about a horizontal axis 20, i.e. pitch, and angular dis- 
placement about a vertical axis 22, i.e. yaw, can be de- 
termined. Optionally, roll about the longitudinal axis 24 
of the device 10 can also be monitored. 
[0032] The invention will be described primarily with 
reference to providing control of a remote element, such 
as controlling a screen cursor of a video display. How- 
ever, it is understood that the approach of tracking atti- 
tude by means of correlating images formed by a pho- 
tosensor array attached to the device has applications 
beyond that of controlling a screen cursor. 
[0033] The sensor array 1 6 and the processing circuit- 
ry 18 are mounted on the front of the controller device 
10 with the array facing outwardly. A lens 26 is shown 
in Fig. 1 as being partially cutaway in order to expose 
the array and circuitry. The lens may be an inexpensive 
plastic lens that is molded to the integrated circuit pack- 
age 12. The focus of the lens is set nominally at infinity. 
The lens provides an image of a region of the surround- 
ing environment to the sensor array 16. A short focal- 
length lens is preferred, since a greater number of fea- 
tures within the environment will be imaged and used 
for the purpose of tracking attitude. The image quality 
demands are sufficiently lax, such that the demands on 
the optical arrangement are not stringent. In fact, a slight 
blurring is desirable, so that the spatial frequencies of 
the image are coarser than the size of the array ele- 
ments. 

[0034] The width of the field of view for imaging the 
environment is a matter of balancing the desire of cap- 
turing as much visible detail as possible with the require- 
ment of avoiding excessive distortion. A 64° field of view 
provides a reasonable compromise between these two 
concerns. The dashed lines 28 in Fig. 1 represent the 
field of view for the sensor array 16. The arrangement 
of the optics and the sensor array may be selected to 
reduce any adverse effects of curvilinear distortion on 
the attitude-tracking approach of the invention. The op- 
tic/sensor arrangement will be described in greater de- 
tail below. 

[0035] On the top surface of the controller device 10 
are a cursor control key 30 and a return key 32. In op- 
eration, the screen cursor of a video display may be cap- 
tured by depression of the control key 30. With the con- 
trol key in a depressed position, the angular displace- 
ment of the device 10 will be translated into movement 
of the cursor along the display screen. For example, in 
selecting a particular movie from a menu of movies pre- 
sented by an interactive television system, key 30 is de- 
pressed and pitch and/or yaw of the device 10 is em- 
ployed to drag the screen cursor to the desired movie. 
The control key 30 is then released to release control of 
the screen cursor. With the cursor indicating the desired 
menu choice, the return key 32 is depressed to activate 
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the selection. Alternatively, a single selection key similar 
in operation to that of a computer mouse may be utilized, 
so that cursor control is continuously enabled. 
[0036] Remote communication is provided by infrared 
transmission via an emitter 34. Infrared emission is well 5 
known in the art. For cursor control, signal vectors are 
generated and transmitted. The vectors are sent as in- 
crements in order to reduce the burden of the lR com- 
munications channel by keeping the data bytes shorter. 
The rate at which vectors are transmitted should be suf- 
ficiently fast to achieve the necessary cursor respon- 
siveness, but not so fast as to tax the limited bandwidth 
of the IR link to the controlled set. As will be explained 
more fully below, in one embodiment the transmission 
is a 1 0-bit word that is emitted every 40 ms, as needed. 
When neither of the keys 30 and 32 is depressed, no 
transmission occurs. This frees the IR emitter for other 
functions during non-cursor events. Thus, the controller 
device 1 0 may include keys to control other events, such 
as the changing of a television channel. 
[0037] Referring now to Figs. 1 and 2, a control clock 
36 determines the timing of operations for the controller 
device 10. The image-capture rate is at least partially 
determined by the features that are to be imaged. Where 
the field of view 28 is a living room of a person engaged 
in operation of an ITV system, the image capture rate 
must be sufficiently great to ensure that the image data 
is sufficiently intelligible to permit correlation between 
successive images. The circuitry of Fig. 2 includes an 
exposure control 38 which shows a minimum exposure 
of 5 \is and a maximum exposure of 5 ms. Electronically, 
the "exposure time" is the interval during which charge 
generated by the individual photosensors is allowed to 
integrate during the 5 ms period between successive im- 
age captures. While the exposure control may be man- 
ually adjusted, the preferred embodiment is one in which 
there is a feedback line 40 from image correlator circuit- 
ry 42 to establish automatic gain control. Thus, if the 
image data is consistently off-scale for performing cor- 
relation processing, the exposure control 38 increases 
the interval for integrating charge by the photosensors. 
[0038] As previously noted , the field of view 28 is con- 
templated to be approximately 64°. In a sensor array of 
32x32 photosensors, a single pixel will have an approx- 
imately 2° field of view (64°/32 pixels). In the preferred 
embodiment to be described below, correlations are lim- 
ited to nearest-neighbor pixel cells. This establishes a 
maximum velocity for movement of the controller device 
12. In the preferred embodiment, the maximum velocity 
should not exceed one pixel length between successive 
image captures. If a frame of image data is to be ac- 
quired at the correlator 42 every 5 ms, the maximum 
velocity will be 200 pixels per second. Since the field of 
view of each pixel is 2°, the maximum velocity is 400° 
per second. 

[0039] The operation of the correlator 42 will be ex- 
plained in detail below. Its basic function is to compare 
the arrangement of pixel values in a first frame of image 



data to the location of a similar arrangement in a sub- 
sequent frame. The sensor array 16 is used to acquire 
a frame of 32x32 pixel values. The first captured frame 
is referred to as a reference frame. The first reference 
frame is captured upon depression of the cursor control 
key 30 of Fig. 1 . A subsequently captured frame of pixel 
values, referred to as a sample frame, is compared with 
the reference frame to detect changes in attitude of the 
controller device 1 0. A new reference frame is then ac- 
quired. The reference frame may be replaced by the 
sample frame after each correlation. Alternatively, the 
reference frame may be replaced with a sample frame 
each time a threshold is reached, such as a time thresh- 
old or a threshold based upon movement of the device. 
[0040] Correlation will be described in detail below. 
Conceptually, the process may be considered as fixing 
the two-dimensional reference frame in position and 
shifting a transparency of the two-dimensional sample 
frame to various positions atop the reference frame in 
order to detect alignment of common features of the two 
frames. This conceptual view is shown in Fig. 5 and is 
computationally carried out by an array of cells such as 
the pixel cell shown in Fig. 6. 

[0041] The correlation processing generates an atti- 
tudinal signal indicative of any changes in angular ori- 
entation of the controller device 10. In Fig. 2, the corre- 
lator 42 is shown as generating a signal in which change 
along the X axis is +0.75 pixels, while the change along 
the Y axis is -0.31 pixels. In Fig. 1, positive movement 
along the X axis may be indicated as a result of angular 
displacement of the device 1 0 such that the field of view 
28 is moved in the direction of arrow 44, while a negative 
movement along the X axis is one in which a component 
of displacement of the field of view 28 is in the direction 
indicated by arrow 46. Regarding the Y axis, a positive 
value at the attitudinal signal indicates manipulation of 
the device 1 0 to rotate the field of view as shown by 
arrow 48, while a negative value of AY indicates dis- 
placement in the direction shown by arrow 50. 
[0042] The preferred approach for operation of the 
correlator 42 is one in which nearest-neighbor pixel 
shifts are executed in order to track changes in attitude 
of the controller device. This approach will be detailed 
when referring to Figs. 4, 5 and 6. However, the attitude 
tracking alternatively may be carried out using correla- 
tions requiring shifts of greater than a single pixel. For 
either approach, interpolations are computationally per- 
formed to identify fractional pixel values. Such image 
correlating interpolating is well understood by persons 
skilled in the art. 

[0043] The output from the correlator 42 is received 
at a multiplier 52 that increases the sensitivity of the cur- 
sor control. In the embodiment of Fig. 2, the multiplier 
doubles the AX and AY values from the correlator. The 
signal is then conducted to an accumulator 54. A divide- 
by-eight circuit 56 dictates operation of the accumulator 
54. Eight samples will be received from the multiplier 52 
before the accumulator outputs a sample to a modulo 
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function block 58. The output of the modulo block will 
be the integer portion of the accumulation of the eight 
samples from the multiplier 52. As previously noted, the 
controller device transmits a cursor control signal by 
means of an IR emitter 34. IR transmission links have 
limited bandwidth. The use of the accumulator 54 reduc- 
es the demands on the communications link. 
[0044] The exemplary sample from the accumulator 
54 in Fig. 2 is shown to be one in which AX = +1 5.2 pixels 
and AY = -3.3 pixels. The modulo block 58 passes only 
the whole numbers to a pulse code modulator 60. Each 
of the AX and AY values is contained within 5 bits, with 
4 bits relating to the numerical value and the remaining 
bit relating to whether the numerical value is positive or 
negative. 

[0045] The values output from the modulo block 58 to 
the modulator 60 are whole number values, while the 
remainder of the sample from the accumulator 54 is con- 
ducted to a residue circuit 62. The residue values are 
saved at the residue circuit and added to subsequent 
values from the accumulator. 

[0046] The remaining arrangement of components is 
well known in the art. A 10-bit word is output from the 
pulse code modulator 60 to an amplifier 64 every 40 ms. 
Thus, the IR transmission from the emitter 34 is only 25 
words per second. The IR transmission is received by 
a sensor 66 of a display system, such as a RIVS. The 
signal from the IR sensor 66 is decoded at a demodu- 
lator 68 and transmitted to a cursor control circuit 70 for 
manipulating a screen cursor of a video screen 72. 
Merely for example, the screen may have 64x64 cursor 
addresses and the screen cursor may move 15x25 = 
375 address states/second. In the example given above 
in which the field of view 28 of Fig. 1 is 64°, the screen 
can be traversed in 0.17 second (64/375). 
[0047] The arrangement and interaction of the com- 
ponents of Fig. 2 permit cursor control based upon track- 
ing of the attitude of the controller device 10 of Fig. 1. 
The controller encodes the device pitch, yaw and op- 
tionally roll by tracking the apparent motion of an image 
of the environment in which the device resides. It is not 
necessary to point the controller in any one direction, 
other than to establish the IR link between the emitter 
34 and the sensor 66. With the possible exception of the 
correlator 42, the individual components of Fig. 2 are 
readily understood by persons skilled in the art and need 
not be explained in detail. However, the preferred em- 
bodiment of the correlator 42 will be set forth below. The 
preferred arrangement of the two-dimensional sensor 
array 16 and the optics 26 will also be set forth. 

COLUMN TRANSFERS OF PHOTOSENSOR 
SIGNALS 

[0048] As previously noted, the embodiment of Figs. 
1 and 2 includes a sensor array 16 having thirty-two col- 
umns and thirty-two rows of photosensors. Referring 
now to Fig. 3, five columns 74, 75, 76, 77 and 78 of the 



thirty-two columns of photosensors are shown. Also 
shown are six of the thirty-two rows 79, 80, 81, 82, 83 
and 84, Each column is operatively associated with a 
separate transfer amplifier 85. A photosensor in a col- 

5 umn is connected to the operatively associated transfer 
amplifier for closing a read switch 86. In the operation 
of the circuitry of Fig. 3, no two photosensors are con- 
nected to the same transfer amplifier simultaneously. 
[0049] Each transfer amplifier 85 includes an input 87 

10 that is connected to a source of a fixed voltage. A second 
input 88 is capacitively connected to the output 90 of the 
transfer amplifier by a transfer capacitor 89. 
[0050] In the operation of the circuit of Fig. 3, the read 
switches 86 of the first row 79 of photosensors may be 

15 closed, so that each transfer capacitor 89 receives a 
charge corresponding to the light energy at the associ- 
ated photosensor in the first row. Thus, the transfer am- 
plifiers 85 and the transfer capacitors 89 function as in- 
tegrators. The received charge is transferred to subse- 

20 quent processing circuitry via the output lines 90. Fol- 
lowing the readout of the first row, the read switches of 
the first row are opened and the transfer amplifiers are 
reset. The read switches of the second row 80 are then 
closed in order to transfer the signals from the photo- 

25 sensors of the second row. The process is repeated until 
each row of photosensors is read. By the operation of 
the transfer amplifiers 85, photosensor signals are 
transferred in a row-by-row fashion to subsequent cir- 
cuitry. 

30 

CORRELATION PROCESSING 

[0051] Fig. 4 illustrates steps for carrying out the cor- 
relation processing for tracking attitude of a device. In 

35 effect, the correlations compare the positions of imaged 
features in successive frames to provide information re- 
lated to changes in angular orientation during time in- 
tervals between frames. In a first step 200, a reference 
frame is acquired. As noted above, the capture of the 

40 reference frame may be initiated by depression of the 
cursor control key 30 on the upper surface of the device 
10 of Fig. 1 . The row-by-row transfer of signals from the 
photosensors continues until each photosensor in the 
array 16 has been read. The reference frame is a frame 

45 of 1 ,024 pixel values (32x32 = 1 ,024). 

[0052] While the tracking of the attitude of the control- 
ler device 1 0 is performed computationally, the concepts 
of the correlation may be described with reference to 
Figs. 4 and 5. A reference frame 202 is shown as having 

50 an image of a T-shaped feature 204. The image feature 
may be a window in a room of a person operating an 
ITV system. At a later time (dt) the sensor array 16 of 
Figs. 1-3 is used to acquire a sample frame 206 as 
shown in Fig. 5. The field of view 28 of the controller 

55 device during acquisition of the sample frame largely 
overlaps the field of view at the time of acquiring the 
reference frame 202. As a result, the imaged feature 
204, e.g. a window, is included within both of the frames. 
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The duration dt and the velocity of the movement of the 
device preferably are such that the common features 
204 will move less than one pixel from the reference 
frame to the sample frame. The schematic representa- 
tion of Fig. 5 shows the feature as having shifted up- 
wardly and to the right by one full pixel. The full-pixel 
shift is assumed only to simplify the representation. 
[0053] Fig. 4 shows the step 208 of acquiring the sam- 
ple frame after the step 200 of acquiring the reference 
frame. A computation step 210 is then executed. The 
correlation step is a shifting of the arrangement of the 
pixel values of the sample frame 204 and two nearest- 
neighbor pixel cells. The shifting is performed sequen- 
tially for each of eight nearest-neighbor pixel cells, as 
represented by member 212 in Fig. 5. Referring to the 
member 21 2, step "0" does not include a shift of the pixel 
values in the sample frame 206, step "1" is a diagonal 
shift upwardly and to the left, step "2" is an upward shift 
of the pixel values of the sample frame, etc. Eight shifts 
of the reference frame represented by member 212 oc- 
cur one at a time, but all pixel values are shifted uniform- 
ly. In this manner, the eight pixel-shifted sample frames 
can be combined with the reference frame 202 to pro- 
duce the frame array 214 of position frames. The posi- 
tion frame designated as "Position 0" does not include 
a shift, so that the result is merely a combination of 
frames 202 and 206. 

[0054] Within the frame array 21 4, "Position 7" has the 
highest correlation. Based upon the correlation results, 
the position of the T-shaped feature 204 in the sample 
frame 206 is determined to be an upward and rightward 
shift relative to the position of the same feature in the 
earlier-acquired reference frame 202. The movement of 
the feature is a result of a combination of pitch and yaw 
of the controller device 10 of Fig. 1, with components 
indicated by the leftward arrow 44 and the downward 
arrow 50. 

[0055] While other correlation approaches may be 
employed, an acceptable approach is a "sum of the 
squared differences" correlation. For the embodiment of 
Figs. 4 and 5, there are nine correlation coefficients (C k 
= C 0 , ... C 8 ) formed from the nine offsets of 212. 
[0056] Correlations are used to find the locations of 
common features 204 of successive frames 202 and 
206 in order to track the angular orientation of the con- 
troller device 10. Summing or integrating angular dis- 
placements from frame-to-frame allows an attitudinal 
signal to be generated for controlling a screen cursor. 
[0057] After the correlation step 210 has been execut- 
ed for the sample frame acquired at step 208, a deter- 
mination is made at step 216 as to whether to maintain 
or replace the reference frame. In some embodiments, 
the reference frame is not replaced. If the same refer- 
ence frame is to be used for subsequent correlation 
computation, the process returns to step 208 and a new 
sample frame is acquired. On the other hand, for appli- 
cations in which there is a substitution of reference 
frames, step 218 shows the use of the previously ac- 



quired sample frame as a new reference frame prior to 
returning to step 208 of acquiring a new sample frame. 
[0058] If the same reference frame is to be used 
throughout a process of dragging a cursor along a 
5 screen, the reference frame can be shifted upon detec- 
tion that common features have traveled more than one 
pixel length since the acquisition of the reference frame. 
Thus, the reference frame is "walked" by detection of 
pitch, yaw and/or roll of the device. This process pro- 
vides a high degree of correlation match, but errors that 
do occur will accumulate if the reference frame is not 
periodically replaced. Thus, in the preferred embodi- 
ment step 21 8 is included so as to place a restriction on 
the growth rate of this "random walk" error. 

COMPUTATION CELL 

[0059] Fig. 6 is a schematic diagram of an individual 
pixel cell, or computation cell 94, for implementing the 
nearest-neighbor shifts of member 212 in Fig. 5 and for 
carrying out the various steps of Fig. 4. However, as will 
be understood by persons skilled in the art, other circuits 
may be used to carry out the process steps described 
and illustrated with reference to Figs. 4 and 5. 
[0060] The computation cell 94 of Fig. 6 is one of an 
array of computation cells. If there is a one-to-one cor- 
respondence of computation cells and photosensors, 
the embodiment of Figs. 1-3 will have 1,024 computa- 
tion cells. For the reference frame, the pixel value from 
the photosensor that corresponds to the particular cell 
94 will be stored at the REFDATA node 148, while pixel 
values of the sample frame will be stored at NN(0) node 
108. The shifted sample frames for forming the frame 
array 214 of Fig. 5 are provided by changing the signal 
connections to NNINPUT input 150 of a square-of-the- 
difference circuit 146. The output 150 of the circuit 146 
is combined with the outputs of the other 1 ,023 compu- 
tation cells to track the attitude of the controller device 
10. 

[0061] Image data WDATA(i) is received from the pho- 
tosensor that corresponds to the computation cell 94 at 
line 96, by means of a charge compensate transistor 
switch 98 under the control of a WR(j) signal. After the 
WR(j) signal is deasserted, the new data is held at ca- 
pacitor 100 and is buffered by an amplifier 102. In the 
entire array of 1,024 computation cells, the array of 
CDATA nodes 104 is collectively referred to as the "ref- 
erence frame" or the "sample frame," depending upon 
the timing of the process. Control input CDOUT 1 06 se- 
lects signal CDATA, comparison data, or selects RE- 
FOUT for the nearest-neighbor output node NN(0) 1 08. 
[0062] The nearest-neighbor inputs NN(0)-NN(8) 
110, 112, 114, 116, 118, 120, 122, 124 and 126 are sep- 
arately selected by means of switch control signals S(0) 
-S(8) on lines 128. The NN(0)-NN(8) inputs 110-126 are 
the outputs of the nearest-neighbor cells according to 
the data cell map 212 of Fig. 5. The switch control sig- 
nals are generated by a 4-9 encoder, not shown, that is 
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external to the computational array. The 4-bit input to 
the encoder is referred to as the nearest-neighbor ad- 
dress and takes on binary values from 0000(0) to 1000 
(8). 

[0063] The nearest-neighbor input (NNINPUT) node 
130 is sampled by pulsing REFLD 132, thereby storing 
the NNINPUT on node REFH 134. Similarly, REFDATA 
1 36 can be sampled and held on REFSH 1 38 by pulsing 
REFSFT 140. 

[0064] For testing, ROWTSTB 142 can be asserted, 
allowing the NN(0) signal to propagate to the TEST out- 
put 144. The TEST signals from each cell in a row of 
cells connect to common vertical buses in each column 
of the computational array and are multiplexed at the 
bottom of the array and driven off-chip. A standard row 
decoder along the left edge of the array allows selection 
of a particular row for test. However, the test feature is 
not critical to the invention. 

[0065] Each computation cell 94 has a circuit 1 46 that 
determines correlation values. A first input 148 receives 
the reference data from REFDATA node 136. A second 
input 150 provides the nearest-neighbor input NNIN- 
PUT selected by the appropriate switch control signal at 
lines 128. The output 152 of the correlation cell is a cur- 
rent. All of the correlation outputs in the computational 
array are summed together in a single off-chip summing 
resistor of a tracking circuit 154. The voltage developed 
across the summing resistor is referred to as the corre- 
lation values in Fig. 4. 

[0066] In the embodiment of Fig. 6, the circuit 146 is 
based on a squared-difference calculation. The cell 94 
may be modified to provide product-based correlations 
without modifying the basic architecture of the array. 
Control inputs S(0)-S(8), REFLD, REFSFT and CDOUT 
are global to the entire array, so that the sample frame 
is manipulated and shifted uniformly within the compu- 
tation cell array. 

[0067] It is important to understand the relationship 
between the nearest-neighbor map represented by 214 
in Fig. 5 for a single cell and for the array as a whole. 
Location 0 of an image refers to the present location of 
the image. When referring to the movement of an image 
from location 0 to location 1 , the representation is that 
the image data in all of the cells of the array are moved 
to the neighbor cell that is to the left and upward. That 
is, the movement is relevant to a single cell in the array 
and is relevant to every cell in the array. 
[0068] The computational array functionality can be 
best described in terms of four basic operations: image 
acquisition, reference image load, correlation computa- 
tion, and reference image translation. The image acqui- 
sition refers to the loading of new image data via the 
WDATA line 96 of each computation cell 94. In the 
present implementation, every 50 ms a new frame of 
pixel values is acquired from the sensor array via the 
column transfer amplifiers. The process of loading a 
new reference image is referred to as a "frame transfer." 
The operations of the computation array described be- 



low are coordinated with the frame transfer process by 
observing and synchronizing with a frame transfer (FTB) 
signal. The validity of a new sample image is signaled 
by the falling edge of the FTB signal. The operations 
5 described below are only appropriate when FTB is not 
asserted. 

[0069] Loading a reference frame of pixel values is re- 
quired before any image correlations can be calculated. 
To load the reference frame, all of the voltages at the 

10 CDATA nodes 104 in the computational array must be 
transferred to the REFH nodes 134. This is accom- 
plished by setting CDOUT 106 and S(0) high, and puls- 
ing the REFLD signal on line 132. The reference frame 
can also be loaded from another nearest-neighbor loca- 
ls tion by asserting one of the S(1 )-S(8) inputs instead of 
S(0). Amplifier 156 buffers the REFH node 134 from the 
REFDATA nodes 1 36. The two-dimensional array of RE- 
FDATA nodes 136 within the computational array are 
collectively known as the reference frame. 

20 [0070] After a reference frame has been loaded, the 
computational array is ready to compute correlations. 
Correlations between the reference frame of pixel val- 
ues and the subsequent sample frame are computed by 
setting the nearest-neighbor address to the desired val- 

25 ue and recording the resulting voltage developed across 
the summing resistor of the attitude tracking circuit 1 54. 
When the sensor array has moved a single pixel dis- 
tance from the location at which the reference frame was 
acquired, a strong correlation will be detected at one of 

30 the nearest-neighbor locations. In Fig. 5, the correlation 
is detected to be at POSITION 7 in the array 214. Sub- 
pixel movements can be determined by interpolating in 
two-dimensional correlation space. 
[0071] It should be noted that correlations between 

35 the reference frame and itself can be computed by set- 
ting CDOUT 106 low and pulsing REFSFT 140. This 
causes the nearest-neighbor inputs to come from the 
reference frame, rather than from the sample frame. 
[0072] Fig. 4 referred to the step 218 of replacing the 

*o reference frame. However, the same reference frame 
may be used for multiple pixel movements by allowing 
the reference to "reach" beyond the nearest-neighbor 
cells. Upon detection that changes in the angular orien- 
tation of the device have caused the field of view to shift 

45 by more than one pixel distance, the reference frame is 
shifted to the new location within the computational ar- 
ray. The shift to the new location is dictated by the 
change in angular orientation of the device. With the ref- 
erence frame in the new location, nearest-neighbor cor- 

50 relations become correlations between the translated 
reference frame and a newly acquired sample frame. By 
translating the pixel values of the reference frame to new 
locations, inter-cell connectivity is kept to a minimum 
without compromising the accuracy of the attitude track- 

55 jng. 

[0073] The translation of the reference frame is ac- 
complished within the individual computation cells of the 
computational array. Referring to the computation cell 
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94 of Fig. 6, a first step is to pulse REFSFT 140 high to 
sample-and-hold the REFDATA signal 136 at REFSH 
138. A third buffer amplifier 158 drives REFOUT 160 
with the REFSH signal. CDOUT 106 is driven low, al- 
lowing REFOUT 1 60 to be the output of the computation 5 
cell 94 at the NN(0) node 108. 
[0074] The nearest-neighbor address is then set to 
select the appropriate S(0)-S(8) input from the switch 
control signal lines 128. For example, if the T-shaped 
feature 204 in Fig. 5 is detected to have moved to the 
right and upwardly from the time at which the reference 
frame 202 is acquired to the time at which the sample 
frame 204 is acquired, the pixel values that comprise 
the reference frame must be shifted from cell-to-cell to 
the right and upwardly. Since S(0)-S(8) signals switch 
the address of the cell input NNINPUT 130, S(7) must 
be driven high to enable movement of the pixel value 
from location 7 of 212 in Fig. 5 to location 0. This takes 
place in each computation cell of the computational ar- 
ray. After REFOUT 1 60 has settled on the NNINPUT 1 30 
of the appropriate neighbor cell, REFLD 132 is pulsed 
high to sample-and-hold the translated reference values 
at REFH 134. After this operation has occurred in each 
of the computation cells of the array, the computational 
array is again ready to compute correlations. 

SENSOR/OPTICS ARRANGEMENT 

[0075] Another feature of the above-described sys- 
tem is the arrangement for significantly reducing the ad- 
verse effects of what is known in lens design as "curvi- 
linear distortions." These distortions are also referred to 
as pincushion, barrel, and perspective distortions. The 
distortions compress rectilinear detail at the outer edges 
of a field of view.. 

[0076] The curvilinear distortions are particularly pro- 
nounced in simple lenses with wide fields of view. Such 
a lens may be used as the tens 26 in Fig. 1 . The distor- 
tions are problematic, since they will cause the apparent 
shape of a feature to change as the feature drifts across 
the field of view. A change in the apparent shape of a 
feature will adversely affect the correlation process de- 
scribed with reference to Figs. 4-6. 
[0077] Fig. 7 shows images of a T-shaped feature, im- 
aged with distortion-free optics onto a 32x32 pixel array 
220. With such optics, the shape of the feature does not 
change with travel across the array, allowing high quality 
correlations and accurate tracking of pitch and yaw. 
[0078] On the other hand, Fig. 8 illustrates the same 
feature imaged onto a pixel array 222 using a simple 
lens. It can be seen that the distortion increases as the 
feature is moved further from the optical axis of the lens. 
Fig. 9 shows in a stylized manner the way in which the 
feature is interpreted by the computational array for per- 
forming the correlation processing of Fig. 5. Each pixel 
in the array 224 of Fig. 9 represents a computational cell 
94, such as the one shown in Fig. 6. It is evident that the 
feature is "viewed" differently as it changes position 



within the computational array. This reduces the quality 
of the correlation, and the accuracy of the tracking of 
device attitude. Equally undesirable, the apparent mo- 
tion of the feature changes with position, further degrad- 
ing the accuracy of attitude tracking. For example, a 
strong off-axis moving detail will be measured as mov- 
ing less than if the identical move were made on-axis. 
The result is a degradation of the attitude-tracking ac- 
curacy of the system. The adverse effects may be made 
smaller by not carrying the reference image across as 
many pixels, but this option of more frequently "zeroing" 
the reference frame increases the number of incremen- 
tal errors, thereby adding its own contribution to tracking 
inaccuracy. 

[0079] Fig. 1 0 illustrates a two-dimensional array 226 
of photosensors configured to match the distortion of the 
lens. That is, the characteristics of the lens system with 
regard to introducing curvilinear distortions are deter- 
mined and the array is configured to compensate for the 
distortions. Projected onto the curvilinear array of Fig. 
10 is the T-shaped feature in three different positions. It 
can be noted that although the images of the feature are 
still distorted, the distortion causes the images to fit with- 
in the pixels. Thus, each feature will be viewed as being 
rectilinear at the computation array level. That is, the 
corner image 228 will be distorted at the sensor level, 
as shown in Fig. 10, but the corner image 230 will be 
"viewed" as being an undistorted "T" at the computation- 
al array 232 of Fig. 11. 

[0080] The method of ascertaining the curvilinear ge- 
ometry of the photosensor array elements may now be 
described. Conceptually, it consists of starting with an 
enlarged object plot of a 32x32 array of rectilinear (not 
curvilinear) cells, perhaps a foot square. The lens of 
choice is next used to produce an image of this array. 
This small and curvilinear image captures the distortions 
of the lens, and represents a reasonable approximation 
to an ideal array geometry. 

[0081] This process produces accurate imaging with 
lenses of any field of view, and has many applications 
such as in positional navigation. However, in the present 
pointing device, it is a goal to transduce angles and not 
distances. For lenses of narrow fields of view, angles 
tend to become increasingly compressed as one moves 
further off of the optical axis. 

[0082] This undesirable angular compression is 
avoided by changing the design process such that a 
segment of a sphere is substituted for the aforemen- 
tioned flat array. The center of the spherical segment is 
overlaid approximately with the nodal point of the lens. 
The segment is lined at intervals of perhaps 2° in latitude 
and longitude. Each cell formed on the spherical seg- 
ment, once imaged and reduced, is taken to be the out- 
line of an array cell. 

[0083] In practice, this method of generating the ge- 
ometry of the curvilinear array is best accomplished us- 
ing computer ray tracing. Many commercial optical de- 
sign programs are available and familiar to those skilled 
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in the art. 
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Claims 

1 . A method of controlling an input to a display screen 
(72) by tracking the attitude of a device (10) com- 
prising the steps of: 

fixing a two-dimensional array (16) of photo- 
sensors to said device; 

attaching optics (26) to said device to image 
features (204) of an environment onto said pho- 
tosensors; 

utilizing said two-dimensional array to acquire 
a first image (202) of features within a field of 
view of said array; 
storing said first image; 

utilizing said two-dimensional array to acquire 
a second image (206) of features within said 
field of view at a time subsequent to acquiring 
said first image, said first and second images 
being images of regions that are largely over- 
lapping such that there are features common to 
both of said first and second images; 
correlating (210, 214) said first and second im- 
ages to detect differences in location of said 
common features within said first and second 
images; 

based upon said step of correlating, forming an 
attitudinal signal indicative of any changes in 
angular orientation of said device during an in- 
terval between acquiring said first and second 
images; and 

transmitting said attitudinal signal from said de- 
vice (10) to a remote receiver (66) associated 
with the display screen (72). 

2. The method of claim 1, wherein the step of input 
control comprises controlling a cursor (70) on the 
display screen (72) in response to said attitudinal 
signal. 

3. The method of claim 2, wherein the location of said 
cursor (70) is controlled by transmitting said attitu- 
dinal signal in a wireless manner. 

4. The method of claim 1 , 2 or 3, wherein said step of 
fixing said two-dimensional array (16) is a step of 
attaching said array to a hand-holdable member. 

5. The method of any preceding claim, wherein said 



steps that include acquiring first and second images 
(202, 206) are steps of forming image data of a re- 
gion of an environment in which said display screen 
(72) resides. 

5 

6. The method of any preceding claim, further com- 
prising periodically acquiring sample images (206) 
and correlating each sample image to an earlier ac- 
quired image (202) to detect differences in location 

10 of said common features within said sample imag- 
es. 

7. A system for tracking attitude and transmitting atti- 
tude information to a display screen (72) compris- 
es ing: 

a housing (10); 

a two-dimensional array (16) of photosensors 
fixed to said housing for forming images of fea- 

20 tures (204); 

memory (94) connected to said array for storing 
a reference frame (202) of pixel values in which 
each pixel value is indicative of light energy re- 
ceived at a particular photosensor at a specific 

25 time; 

a correlator (42) connected to said array and to 
said memory for correlating an arrangement of 
said pixel values of said reference frame with 
at least one sample frame (206) of pixel values 

30 formed in response to light energy received at 

said array subsequent to said specific time; 
a computational arrangement (150, 154), re- 
sponsive to detection of correlations by said 
correlator, for generating an attitudinal signal 

35 representing pitch and yaw of pixel values of 

said at least one sample frame relative to said 
pixel values of said reference frame, said atti- 
tudinal signal thereby representing pitch and 
yaw of said housing; and 

40 means for supplying the attitudinal signal to a 

remote receiver (66) constituting an input to the 
display screen (72). 

8. A system according to claim 7, wherein said attitu- 
45 dinal signal from said receiver (66) is arranged to 

control a cursor (70) on the display screen (72). 

9. The system of claim 8, further comprising a demod- 
ulator (68) connected to receive said attitudinal sig- 

50 nal from said receiver (66) and for forming a cursor- 
control signal for manoeuvring the cursor (70) on 
the display screen. 

10. The system of any of claims 7 to 9, wherein said 
55 housing (1 0) is hand holdable. 
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Patentanspriiche 

1. Ein Verfahren zum Steuern einer Eingabe in einen 
Anzeigebildschirm (72) durch Verfolgen der Stel- 
lung einer Vorrichtung (10), das folgende Schritte 
aufweist: 

Befestigen eines zweidimensionalen Arrays 
(16) aus Photosensoren an der Vorrichtung; 

Anbringen einer Optik (26) an der Vorrichtung, 
um Merkmale (204) einer Umgebung auf die 
Photosensoren abzubilden; 

Verwenden des zweidimensionalen Arrays, um 
ein erstes Bild (202) von Merkmalen in einem 
Sichtfeld des Arrays zu erlangen; 

Speichern des ersten Bildes; 

Verwenden des zweidimensionalen Arrays, um 
ein zweites Bild (206) von Merkmalen in dem 
Sichtfeld zu einem Zeitpunkt, der auf ein Erlan- 
gen des ersten Bildes folgt, zu erlangen, wobei 
das erste und das zweite Bild Bilder von Regio- 
nen sind, die sich stark uberlappen, so dafi 
Merkmale vorliegen, die sowohl dem ersten als 
auch dem zweiten Bild gemein sind; 

Korrelieren (210,214) des ersten und des zwei- 
ten Bildes, um Unterschiede bezuglich einer 
Position der gemeinsamen Merkmale in dem 
ersten und dem zweiten Bild zu erfassen; 

auf der Basis des Schritts des Korrelierens, BW-. 
den eines stellungsbezogenen Signals, das et- 
waige Anderungen der Winkelausrichtung der 
Vorrichtung wahrend eines Intervalls zwischen 
einem Erlangen des ersten und des zweiten Bil- 
des anzeigt; und 

Ubertragen des stellungsbezogenen Signals 
von der Vorrichtung (10) zu einem Fernempfan- 
ger (66), der dem Anzeigebildschirm (72) zuge- 
ordnet ist. 

2. Das Verfahren gemafi Anspruch 1, bei dem der 
Schritt einer Eingabesteuerung ein Steuern eines 
Cursors (70) an dem Anzeigebildschirm (72) als 
Antwort auf das stellungsbezogene Signal auf- 
weist. 

3. Das Verfahren gemaft Anspruch 2, bei dem die Po- 
sition des Cursors (70) durch ein Ubertragen des 
stellungsbezogenen Signals auf eine drahtlose 
Weise gesteuert wird. 

4. Das Verfahren gemali einem der Anspruche 1-3, 



bei dem der Schritt des Befestigens des zweidimen- 
sionalen Arrays (16) ein Schritt eines Anbringens 
des Arrays an einem in der Hand haltbaren Bau- 
glieds ist. 

5 

5. Das Verfahren gemaft einem der vorhergehenden 
Anspruche, bei dem die Schritte, die ein Erlangen 
eines ersten und eines zweiten Bildes (202, 206) 
umfassen, Schritte eines Bildens von Bilddaten ei- 

10 ner Region einer Umgebung sind, in der sich der 
Anzeigebildschirm (72) befindet. 

6. Das Verfahren gemali einem der vorhergehenden 
Anspruche, das ferner ein periodisches Erlangen 

15 von Musterbildem (206) und ein Korrelieren jedes 
Musterbildes mit einem fruheren erlangten Bild 
(202), um Unterschiede bezuglich einer Position 
der gemeinsamen Merkmale in den Musterbildem 
zu erfassen, aufweist. 

20 

7. Ein System zum Verfolgen einer Stellung und zum 
Ubertragen von Stellungsinformationen zu einem 
Anzeigebildschirm (72), das folgende Merkmale 
aufweist: 

25 

ein Gehause (10); 

ein zweidimensionales Array (16) aus Photo- 
sensoren, das an dem Gehause befestigt ist, 
30 zum Bilden von Bildern von Merkmalen (204); 

einen Speicher (94), der mit dem Array verbun- 
den ist, zum Speichern eines Referenzrah- 
mens (202) von Pixelwerten, bei dem jeder Pi- 
35 xelwert eine zu einem spezifischen Zeitpunkt 

bei einem bestimmten Photosensor empfange- 
ne Lichtenergie anzeigt; 

einen Korrelator (42), der mit dem Array und 
40 dem Speicher verbunden ist, zum Korrelieren 

einer Anordnung der Pixelwerte des Referenz- 
rahmens mit mindestens einem Musterrahmen 
(206) von Pixelwerten, die als Antwort auf eine 
nach dem bestimmten Zeitpunkt bei dem Array 
45 empfangene Lichtenergie gebildet sind; 

eine Rechenanordnung (1 50, 1 54), die auf eine 
Erfassung von Korrelationen durch den Korre- 
lator anspricht, zum Erzeugen eines stellungs- 

50 bezogenen Signals, das einen Abstand und ei- 

ne Drehung von Pixelwerten des mindestens 
einen Musterrahmens bezuglich der Pixelwerte 
des Referenzrahmensdarstellt, wobei das stel- 
lungsbezogene Signal dadurch einen Abstand 

55 und eine Drehung des Gehauses darstellt; und 

eine Einrichtung zum Zufuhren des stellungs- 
bezogenen Signals zu einem Fernempfanger 
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(66), der eine Eingabe zu dem Anzeigebild- 
schirm (72) bildet. 

8. Ein System gemaft Anspruch 7, bei dem das stel- 
lungsbezogene Signal von dem Empfanger (66) an- 
geordnet ist, urn einen Cursor (70) auf dem Anzei- 
gebildschirm (72) zu steuern. 

9. Das System gemafi Anspruch 8, das ferner einen 
Demodulator (68) aufweist, der verbunden ist, urn 
das stellungsbezogene Signal von dem Empfanger 
(66) zu empfangen, und zum Bilden eines Cursor- 
steuersignals zum Manovrieren des Cursors (70) 
auf dem Anzeigebildschirm. 

10. Das System gemafi einem der Anspruche 7 bis 9, 
bei dem das Gehause (10) in der Hand haltbar ist. 



Revendications 

1 . Procede de commande d'une entree vers un ecran 
d'affichage (72) par suivi de I'attitude d'un dispositif 
(10) comprenant les etapes consistant a : 

fixer une matrice bidimensionnelle (1 6) de pho- 
todetecteurs audit dispositif ; 
fixer des elements optiques (26) audit dispositif 
pour former des images d'entites (204) d'un en- 
vironnement sur lesdits photodetecteurs ; 
utiliser ladite matrice bidimensionnelle pour ac- 
quehr une premiere image (202) d'entite dans 
un champ de vision de ladite matrice ; 
memoriser ladite premiere image ; 
utilisee ladite matrice bidimensionnelle pour 
acquerir une seconde image (206) d'entit6 
dans ledit champ de vision a un instant qui suit 
I'acquisition de ladite premiere image, lesdites 
premiere et seconde images etant des images 
de regions se recouvrant largement de fagon 
qu'il existe des entites communes a la fois audi- 
tes premiere et seconde images ; 
effectuer une correlation (210, 214) desdites 
premiere et seconde images pour detecter les 
differences d'emplacement desdites entites 
communes dans lesdites premiere et seconde 
images ; 

en se basant sur ladite etape de correlation, for- 
mer un signal d'attitude indiquant de quelcon- 
ques variations d'orientation angulaire dudit 
dispositif pendant un intervalle entre I'acquisi- 
tion desdites premiere et seconde images ; et 
transmettre ledit signal d'attitude depuis ledit 
dispositif (10) a un recepteur distant (66) asso- 
cie a Pecran d'affichage (72). 

2. Procede selon la revendication 1 , dans lequel I'eta- 
pe de commande d'entree comprend la commande 



d'un curseur (70) sur I'ecran d'affichage (72) en re- 
ponse audit signal d'attitude. 

3. Procede selon la revendication 2, dans lequel I'em- 
5 placement dudit curseur (70) est commande en 

transmettant ledit signal d'attitude d'une maniere 
sans fit. 

4. Procede selon la revendication 1, 2 ou 3, dans le- 
10 quel ladite etape de fixation de ladite matrice bidi- 
mensionnelle (16) est une etape de fixation de la- 
dite matrice a un element pouvant etre tenu a la 
main. 

15 5. Procede selon I'unequelconque des revendications 
precedentes, dans lequel lesdites etapes compor- 
tant I'acquisition des premiere et seconde images 
(202, 206) sont des etapes de formation de don- 
nees d'images d'une region d'un environnement 

20 dans lequel se trouve ledit ecran d'affichage (72). 

6. Procede selon I'une quelconque des revendications 
precedentes, comprenant en outre I'acquisition pe- 
riodique d'images echantillon (208) et la correlation 
25 de chaque image echantillon avec une image ac- 
quise precedemment (202) pour detecter les diffe- 
rences d'emplacement desdites entites communes 
dans lesdites images echantillon. 

30 7. Systeme de suivi d'attitude et de transmission d'in- 
formations d'attitude a un ecran d'affichage (72) 
comprenant : 

un boitier (10) ; 
35 une matrice bidimensionnelle (1 6) de photode- 

tecteurs fixee audit boitier pour former des ima- 
ges d'entites (204) ; 

une memoire (94) connectee a ladite matrice 
pour stocker une image de reference (202) de 
40 valeurs de pixels dans laquelle chaque valeur 

de pixel indique une energie lumineuse recue 
au niveau d'un photodetecteur particulier a un 
moment specifique ; 

un correlateur (42) connecte a ladite matrice et 
45 a ladite memoire pour effectuer la correlation 

d'un agencement desdites valeurs de pixels de 
ladite image de reference avec au moins une 
image echantillon (206) de valeurs de pixels 
forme en reponse a I'energie lumineuse recue 
so au niveau de ladite matrice a la suite dudit mo- 

ment specifique ; 

un agencement de calcul (1 50, 1 54) reagissant 
a la detection des correlations par ledit corre- 
lateur, pour produire un signal d'attitude repre- 
55 sentant le tangage et le lacet des valeurs de 

pixels dudit au moins une image echantillon par 
rapport auxdites valeurs de pixel de ladite ima- 
ge de reference, ledit signal d'attitude repre- 
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sentant ainsi le tangage et le lacet dudit boitier ; 

des moyens pour fournir le signal d'attitude a 
un recepteur distant (66) constituant une entree 
vers I'ecran d'affichage (72). 5 

8. Systeme selon la revendication 7, dans lequel ledit 
signal d'attitude provenant dudit recepteur (66) est 
agence pour commander un curseur(70) sur I'ecran 
d'affichage (72). 10 

9. Systeme selon la revendication 8, comprenant en 
outre un demodulateur (68) connecte pour recevoir 
ledit signal d'attitude depuis ledit recepteur (66) et 
pour former un signal de commande de curseur is 
pour manoeuvrer le curseur (70) sur I'ecran d'affi- 
chage. 

10. Systeme selon Tune quelconque des revendica- 
tions 7 a 9, dans lequel ledit boitier (10) peut etre 20 
tenu a la main. 
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